There are two schools of thought on this. One is that you should not really consider purchasing your personal gear until you are sure what type of diving you like. This school believes you should buy only mask, fins and snorkel, for fit and sanitary reasons.
The other school of thought is that rental gear, especially in international tropical locations, is often second rate and poorly maintained, and that gear you purchase will be better and more reliable.
Also once you have some idea about what you need, it is always recommended you buy the best quality you can afford. Scuba equipment is after all, life support equipment.
Most start with mask, fins, snorkel, then probably move to a wetsuit, then regulator and BCD. Typically people agree a cylinder is probably the last piece of equipment to buy.
Source: http://scifi.squawk.com/scuba.html (adapted by Sue Crowe)